Field of the Invention
The present invention relates generally to the field of interactive multimedia broadcasting, and more particularly so the automatic detection and synchronization of a multimedia content broadcast and then reproduced on a first device, for example a television, so as to allow the display of an enriched content on a second screen with which a digital tablet, a telephone or a fixed or portable PC is equipped.
Description of Related Art
Starting from the observation that a significant and growing percentage of viewers consults the screen of their portable device or their computer (hereinafter referred to as “second screen” and “second device”) while watching a program broadcast on their main television, numerous applications to be loaded and executed on the second device for display on the second screen are today developed so as to make it possible to enrich the information relating to the program broadcast and to offer viewers ample possibilities of interaction with the program.
By way of nonlimiting example, during a televised game show broadcast on a channel, the viewer can display on his second screen the same questions as those which are posed to the participants, answer them, compare his scores, dispatch real-time comments, or any other action which truly gives the user the impression of participating in the transmission.
Applications of this type are also of interest to advertisers who see an opportunity to provide more information relating to a product forming the subject of a television commercial break.
From the user's point of view, these experiences, also called “second-screen. experiences”, rely on the use of a system architecture making it possible to feed the second screen (or companion screen) with the enriched contents during the broadcasting of a program.
The Applicant has in particular developed the PLAYALONG (registered trademark) system, the architecture of which hinges around three main parts:                at the level of the “back office” (term used to indicate the components and applications of the Internet which are not visible to the end user), a CMS tool (the initials standing for Content Management System) which will essentially prepare, format and store all the enriched contents that it is desired to dispatch, and decide the instants at which these enriched contents must be dispatched;        the PLAYALONG platform proper, comprising a plurality of servers hosted in a computing cloud so as to recover the enriched contents dispatched by the CMS tool, and whose role is to redeploy these contents to a large number of connected mobile devices for display on their screen.        The client applications loaded and executed on the end users' mobile devices so as to receive the enriched contents dispatched by the platform and display them on the screen associated with these devices.        
The PLAYALONG platform uses, in particular, push servers to make it possible to push the contents (example: quiz, survey, video, image, score table) to the mobile devices without the latter having to ask for them.
It is readily understood that certain enriched contents must appear in a manner perfectly synchronized with the main multimedia content displayed on the first screen. A defect of synchronization would indeed have a negative impact on the second-screen experience.
Now, at the so-called reference instant at which a program with a multimedia content to which it is desired to append an enriched content is actually broadcast, a highly variable lag may elapse before this multimedia content is actually displayed on the first screen, this lag being dependent on the broadcasting network making it possible to reach the main television. Thus, a user whose television receives a multimedia program via digital terrestrial television will not view the images of this same multimedia program at the same time as a user whose television is connected via an Internet box or else via a satellite network. The offsets between networks can be fairly significant; up to as much as 20 seconds.
Synchronization solutions using the recognition, by the mobile device fitted with the second screen, of the sound which passes on the main device have already been proposed. These solutions using this principle of ACR (the initials standing for Audio Content Recognition) are of two types:
A first technique, known as “Watermarking”, consists in periodically injecting, into the audio band of the televised program to be broadcast, a particular audio signal (or mark) physiologically inaudible to the human ear, comprising the reference instant, that is to say the instant of transmission of the broadcast image. When the audio band is reproduced by the loudspeakers of the main device, for example the television, a specific application loaded and executed by the second device will also listen to this audio band, in particular via the microphone integrated into the mobile device, recognize the hidden marks and extract therefrom the reference instant. The second device, which comprises its own internal clock, will then be able to determine the temporal offset between the reference instant and the instant at which the corresponding program is actually reproduced on the television, and use this temporal offset to determine the precise instants at which enriched contents must be displayed on the second screen. In the case of live transmissions, this “Watermarking” technique is very constraining for the production facilities of television channels which must be equipped with the appropriate hardware making it possible to inject the hidden audio marks in real time. Furthermore, the bandwidth used for these marks is low, thus precluding the transport of a great deal of information.
According to another technique, known as “Fingerprinting”, and described in particular in document WO 2013/040533 A1, a specific application loaded and executed on the second device will allow it so listen, via its integrated microphone, to the audio band reproduced by the loudspeakers of the main device, and to tap off audio samples that it will transmit to a remote platform, typically an Internet server. For its part, the remote platform listens to and stores permanently all the programs such as broadcast on all the channels. When it receives an audio sample tapped off by the second device, the platform is therefore able to compare this sample with the programs listened to, and if it finds it, to calculate the temporal offset that it will be able to transmit in return so the second device. The “Fingerprinting” solution is less constraining for the broadcasters of multimedia programs since no signal needs to be injected into the broadcast audio/video signal. Nonetheless, the devices on the server platform side for listening to the programs and responding to significant loadings from users are expensive. Likewise, the processing times at the level of the platform in order to be able to perform the comparisons of all the samples received with the set of programs listened to can be significant, so that there is a risk of the second device not receiving on time the item of information regarding the temporal offset that it must apply to control the display of certain enriched contents on the second screen, or reacting too late.